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The  effect  of  predictive  solutions  on  human  onerator  training 
time  and  post-training  performance  in  a  complex  manual  control  system 
has  been  investigated.  A  control  system  with  a  slow  and  complex  response 
to  the  input  signals  was  formulated.  Fifty  operators,  25  with  the  aid 
of  nredictive  solutions  and  25  without  the  predictive  solutions,  were 
tested  and  the  mean  performance  of  each  group  was  compared  to  that 
of  the  other. 

There  was  a  significant  improvement  in  the  training  time  when  the 
predictive  solutions  were  provided.  The  improvement  was  directly  pro- 
portional to  the  complexity  of  the  system.  The  greater  the  initial 
challenge  of  the  system,  the  greater  was  the  worth  of  the  predictive 
solutions.  The  post-training  performance  was  better  by  a  factor  of  two 
when  the  predictive  solutions  were  available  to  the  operator.  The 
performance  of  the  operators  without  the  aid  of  predictive  solutions 
remained  inconsistent  even  after  a  steady-state  performance  had  been 
reached. 
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I.   INTRODUCTION 

The  need  for  highly  competent  human  operators  in  vehicular  control 
and  naviqation  systems  has  become  pronounced  in  today's  technology. 
The  military  and  the  aerospace  program  in  particular  involve  human  oo- 
erators  in  many  complex  control  systems.  Missile  guidance  and  spacecraft 
control  are  prime  examples  of  this  type  of  system. 

In  a  complex  manual  control  system  with  a  lonq  training  period, 
a  human  operator  should  be  able  to  attain  a  competent  level  of  perfor- 
mance in  a  shorter  time  interval  with  the  aid  of  predictive  solutions. 
After  the  operator's  training  is  completed,  he  should  then  be  able  to 
perform  as  well  without  the  predictive  solutions,  but  in  more  complex 
systems  it  might  prove  necessary  to  utilize  the  predictive  solutions 
in  order  to  maintain  an  acceptable  performance  level.  In  1954  Ziebolz 
and  Paynter  discussed  the  possibility  of  two-time  scale  automatic  control 
systems  in  which  predictive  information  about  the  future  performance  of 
the  process  under  control  was  computed  [Ref.  4].  In  1963,  simulating 
space  ship  moon  landings,  L.C.  Fargel  and  E.A.  Ulbrich  proved  that 
predictive  display  markedly  improved  the  learning  period  when  using 
simple  inputs.  The  performance  measure  Fargel  and  Ulbrich  tested  was 
the  quantity  of  fuel  consumed.  Their  work  is  in  Ref,  3. 

For  predictive  solutions  to  be  of  value,  the  system's  response 
needs  to  be  a  slow  and  complex  function  of  the  input  signal.  If  this 
is  not  true,  the  operator  can  easily  master  the  system  without  the 
predictive  solutions,  and  the  predictive  solutions  would  be  a  wasted 
expense.  Also  the  response  of  the  system  should  not  be  easilv 


describable  by  the  operator.  More  work  needs  to  be  done  in  this  field 
in  order  to  establish  exactly  when  a  predictive  solution  system  can 
be  employed  economically  and  practically,  and  effectively  reduce  the 
traininq  time  of  an  operator,  as  well  as  improve  his  oost-traininq 
performance. 
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II.  NATURE  OF  THE  PROBLEM 

The  objective  of  the  problem  was  to  investiqate  the  use  of  nre- 
dictive  solutions  in  the  training  and  performance  of  human  operators. 
The  nroblem  was  formulated  such  that  the  actual  training  Deriod  and  the 
post-training  performance  were  of  primary  consideration.  Fuel  consump- 
tion and  other  criteria  were  completely  disregarded.  A  hybrid  computer 
was  used  for  control  and  simulation  of  the  real-time  and  fast-time  systems 

A  complex  manual  control  system  whose  response  to  the  input  controls 
was  slow  and  not  readily  learned  by  a  human  operator  had  to  be  formu- 
lated. The  system  selected  was  a  two-axis  system  in  which  the  operator 
was  reguired  to  place  a  movable  Doint  as  close  as  possible  to  a  target 
at  the  end  of  a  specified  time  interval.  The  first  candidate's  dynamics 
(see  Fig.  1)  were  found  unsatisfactory;  the  response  of  the  system  to 
the  input  control  was  too  fast  and  not  complex  to  any  degree.  For  the 
second  candidate  (see  Fig.  2)  the  response  was  slow,  particularly  when 
the  velocity  of  the  point  was  large,  and  was  a  more  complex  function  of 
the  input.  This  system  was  selected. 

Seven  trial  human  operators  were  then  observed  and  each  was  qiven 
different  magnitudes  of  control.  It  was  discovered  that  the  magnitude 
of  the  control  had  to  be  kept  small.  This  was  so  because  if  the  control 
was  too  qreat,  the  operator  could  easily  overcome  any  complexity  in  the 
system  by  sheer  force.  When  the  control  was  made  too  small,  however, 
it  became  impossible  for  the  operator  to  accomplish  his  mission.  By 
comparing  the  performance  of  the  operators  tested,  a  degree  of  control 
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was  selected  such  that  after  the  operator  had  learned  the  system  he 
could  easily  achieve  his  goal  no  matter  what  the  initial  conditions 
were,  but  also  such  that  the  response  was  still  slow  and  complex. 
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III.  THE  CONTROL  SYSTEM 

A  block  diagram  of  the  entire  system  is  shown  in  Fig.  3.  Each 
section  is  fully  explained  below. 

The  display  consists  of  an  oscilloscope  which  displays  two  separate 
solutions,  the  fast-time  and  the  real-time.  The  operator's  view  of  the 
scope  is  shown  in  Fig.  4. 

The  next  section  consists  of  the  human  operator.  His  purnose  was 
to  observe  the  display  and  adjust  the  controls  as  needed. 

The  control  section  is  made  up  of  a  chair  with  a  control  stick 
mounted  on  the  right  arm.  Forward  and  backward  motion  on  the  stick 
corresponded  to  forces  in  the  +Y  and  -Y  directions,  respectively.  Right 
and  left  motion  on  the  stick  corresponded  to  forces  in  the  +X  and  -X 
directions,  respectively.  The  magnitude  of  the  forces  was  linear  be- 
tween extremes  in  movement  of  the  controls. 

The  real-time  segment  is  the  next  component  of  the  system.  It  is 
comprised  of  two  sections,  the  digital  and  the  analog.  A  schematic 
of  the  analog  section  is  shown  in  Fig.  2.  A  schematic  of  the  analog 
logic  control  is  displayed  in  Fig.  5.  The  X  and  Y  systems  were  ident- 
ical except  that  the  initial  condition  of  the  Y  integrator  was  always 

2 
zero.  The  transfer  functions  of  X  and  Y  were  1/S  .  The  X  and  Y  forces 

were  the  only  inDuts  to  the  system  other  than  the  initial  conditions. 

A  compute  time  of  sixteen  seconds  was  selected.  Shorter  (faster)  tasks 

are  outside  the  realm  of  human  ability  to  follow,  react,  or  anticipate. 

Sixteen  seconds  is  a  reasonable  length  both  from  a  human  point  of  view  and 


13 


>- 
o 


o 
_i 

CD 


CO 


14 


7N 


It- 


o 

CM 


CVJ 

E 


00 

I— 
U- 


._  o 


o 
o 

o 


m 
o 


O 

CM 


o  M 


a; 
o 

h- 
<c 

OL 
UJ 

a. 
o 


cn 


o 
i 


15 


^o-c*-1-4 


O 
1-4 

O 


o 
on 

o 
o 

o 

-J 
<c 

<t 

u_ 
o 

o 


CD 


16 


from  computer  simulation  time  requirements.  The  outnut  of  the  X  and  Y 
integrators  on  the  analoq  comnuter  were  continuously  fed  to  the  display 
section. 

The  analoq  logic  control  was  designed  to  provide  accurate  results 
for  the  predictive  solutions,  to  yield  the  real-time  compute  cycle,  and 
to  allow  the  display  of  the  two  solutions  simultaneously.  The  real-time 
system  was  operated  in  clock  mode,  with  sixteen  seconds  in  comnute  and 
then  one-half  second  each  in  reset  and  hold.  The  logic  for  the  displav 
is  explained  in  Appendix  A.  The  fast-time  control  consisted  of  two 
counters,  a  400-decimal  counter  and  a  four-decimal  counter.  The  400 
counter  was  driven  by  a  100  cycle  per  second  clock  which  yielded  a  four- 
second  clock.  This  counter  was  a  preset  counter  which  was  reset  each 
time  400  counts  were  reached.  By  the  use  of  nand  gates,  the  loqic  was 
constructed  such  that  two  flip-flops  were  driven  by  Dulses  which  were 
true  when  either  compute,  reset,  or  hold  were  needed  on  the  fast-time 
integraters.  The  outputs  of  these  flin-flons  were  inverted  and  innuted 
to  two  drivers,  which  in  turn  went  to  the  inteqraters.  These  fast-time 
integraters  were  100  times  faster  than  the  real-time  integraters.  The 
400  counter  drove  the  four  counter  once  each  cycle.  This  counter  was 
needed  in  order  to  readjust  the  compute  time  to  account  for  the  time 
already  expired  in  the  real-time  cycle. 

The  digital  section  had  four  functions.  The  first  was  control  of 
the  analog  computer.  The  second  function  was  the  selection  of  random 
values  for  the  initial  conditions  of  X,  the  velocity  of  X,  and  the  ve- 
locity of  Y.  The  recording  of  the  final  value  of  X  and  Y  was  the 
third  purpose.  The  last  task  of  the  digital  computer  was  to  calculate 
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the  error  for  each  run,  the  challenge  level  of  the  initial  conditions, 
and  to  store  this  data  on  maqnetic  tape  for  later  analysis.  A  flow 
chart  of  the  main  Droqram  is  shown  in  Fig.  6. 

The  initial  conditions  were  uniformly  distributed  independent 
random  variables  with  ranges  as  follows: 

0.0  <  X(0)  <  10.0 
-1.25  <  X(0)  <_  1.25 
0.0  <_  Y(0)  <_  1.25 
These  limits  were  selected  for  two  reasons,  the  first  beinq  the  elim- 
ination of  scalinq  problems  on  the  analoq  computer  and  the  second 
beinq  the  need  to  have  the  first  fast-time  solution  aDDear  on  the  scope. 
The  initial  conditions  were  determined  randomly  in  order  to  increase 
the  challenge  of  the  control  problem  to  the  human  operator,  and  to 
eliminate  the  possibility  of  rapid  adaptation  to  a  single  set  of  initial 
conditions  by  any  operator. 

The  next  segment  of  the  control  system  was  the  fast-time  solution. 
The  fast-time  system  was  exactly  identical  to  the  real-time  system  except 
that  it  was  run  100  times  faster.  There  was  no  input  to  the  fast-time 
system  other  than  initial  conditions.  The  initial  conditions  were  taken 
directly  from  the  real-time  system,  X(0)  from  the  X  output  of  the  real- 
time svstem,  etc. 

There  were  four  fast-time  solutions  displayed  at  four-second  intervals 
during  the  sixteen  second  real-time  compute  cycle.  The  first  was  shown 
at  0.16  seconds,  the  second  at  4.12  seconds,  the  third  at  8.08  seconds, 
and  the  fourth  at  12.04  seconds.  Each  was  displayed  until  one  second 
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before  the  next  fast-time  solution  was  to  be  computed.  At  that  time  the 
fast-time  system  went  into  reset  for  one  second.  Granhs  of  the  comnute, 
reset,  and  hold  cycles  are  displayed  in  Fig.  8. 

The  fast-time  display  showed  the  human  operator  exactly  what  the 
state  of  the  system  would  be  at  the  final  time  (t  =  16  seconds)  if  he 
applied  no  further  controls  throughout  the  rest  of  the  real-time 
solution.  This  enabled  the  ooerator  to  see  and  determine  the  direction 
in  which  to  move  the  control  stick  and  to  estimate  how  much  control 
he  should  apply. 
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IV.  EXPERIMENTAL  PROCEDURE 

In  order  to  obtain  meaningful  data  a  large  number  of  human  operators 
had  to  be  tested.  To  enable  investigation  of  improvement  in  training 
times,  egual  numbers  of  operators  using  predicted  solutions  and  not 
using  predicted  solutions  had  to  be  used.  It  was  decided  to  have  50 
operators,  25  with  fast-time  solutions  displayed  and  25  without  the  aid 
of  fast-time  solutions. 

Of  the  oDerators  tested,  there  were:  23  graduate  students,  all 
of  whom  were  naval  officers,  22  electronic  laboratory  technicians,  two 
computer  suDervisors,  one  professor  in  electrical  engineering,  and  two 
computer  technicians.  Each  operator  made  100  separate  runs  of  the 
Droblem.  This  number  was  decided  upon  as  a  result  of  observing  the 
seven  trial  operators,  whose  average  training  period  was  about  80  runs. 

No  operator  was  allowed  any  trial  runs.  Each  operator  was  given 
the  following  information:  the  objective  is  to  be  at  the  target  at  the 
end  of  16  seconds;  there  will  be  100  runs  with  one-second  reset  time 
between  runs;  the  operation  of  the  controls  was  explained;  the  response 
is  slow;  the  initial  conditions  are  random.  The  operators  who  were 
given  the  fast-time  solutions  were  also  explained  their  meaning  and 
method  of  display. 
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V.  ANALYSIS  OF  DATA 

The  final  position  at  the  end  of  each  run  and  the  initial  challenqes 
were  the  only  data  used  to  analyze  the  performance  of  the  human  ODerators. 
The  final  position  was  used  to  compute  the  error  of  the  run.  The  error 
was  defined  as  follows: 

ERR0R(K)=(X-5)2  +  (Y-10)2.  (1) 

This  is  the  square  of  the  distance  from  the  final  position  to  the  target. 
It  was  squared  because  negative  error  was  considered  to  be  of  consequence 
equal  to  that  of  positive  error.  The  error  of  each  run  was  stored  on 
magnetic  tape  so  it  could  be  later  orocessed. 

In  order  to  reinitialize  the  subroutine  that  produced  the  random 
initial  conditions,  it  was  necessary  to  recomnile  the  main  Drogram  imme- 
diately before  each  ooerator  beqan  his  runs.  Each  ooerator  therefore, 
was  presented  with  exactly  the  same  sequence  of  sets  of  initial  conditions 
as  was  every  other  operator.  This  allowed  for  direct  comparison  of 
all  human  operators. 

Due  to  the  random  initial  conditions,  the  difficulty  of  control 
varied.  The  level  of  difficulty  was  called  the  initial  challenge.  Two 
types  of  challenge  were  present.  These  were  an  initial  position  chal- 
lenge and  an  initial  velocity  challenge. 

There  were  five  degrees  of  initial  position  challenge.  The  least 
challenge  (first  degree)  was  considered  to  be  that  case  where  the  initial 
position  in  X  was  in  the  center  of  its  ranne,  or  in  other  words,  directly 
under  the  target.  The  degree  of  challenge  increased  as  the  initial 
position  moved  to  either  of  its  extremes  (fifth  degree  was  designated 
the  greatest  challenge).  The  challenge  sectors  are  shown  in  Fig.  7. 
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The  other  type  of  initial  challenge  was  associated  with  velocity. 
A  velocity  challenge  was  defined  to  exist  when  the  magnitudes  of  both 
X(0)  and  Y(0)  were  greater  than  one-half  their  maximum  possible  value. 

After  the  50  operators  had  been  tested,  the  data  was  kept  in  two 
separate  groups,  the  fast-time  operators  and  the  real-time-only  operators. 
The  mean  value,  the  variance,  and  the  standard  deviation  for  each  of 
the  100  runs  for  each  group  of  25  were  calculated  as  follows: 

MEAN(K)=»[z25  ERR0R(K,KK)]/25.0  (2) 

KK=1 

VAR(K)  =  Z25  [ERROR(K,KK)-MEAN(KD2  (3) 

KK=1 

STDEV(K)=[VAR(K)]1/2  (4) 

The  maximum  error  and  the  minimum  error  for  each  run  were  also  computed. 

Graphs  of  every  possible  combination  of  the  velocity  and  position 
challenges  were  then  plotted.  These  graphs  each  contained  five  sepa- 
rate curves.  The  abscissa  for  all  five  curves  was  the  run  number,  K. 
The  five  ordinates  were  the  maximum  error,  the  mean  value  plus  the 
standard  deviation,  the  mean  value,  the  mean  value  minus  the  standard 
deviation,  and  the  minimum  error.  These  same  five  curves  were  also 
plotted  in  a  summary  graph  which  included  all  100  runs  no  matter  what 
the  initial  condition  was.  Another  plot  of  the  mean  value  versus  the 
run  number  for  all  runs  was  made.  This  made  a  total  of  19  graphs  per 
group.  These  38  graphs  were  then  examined  to  observe  whether  or  not 
the  predictive  solutions  aided  the  operator  and  if  so  how  much.  The 
possibility  of  different  training  times  for  the  various  combinations 
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of  initial  conditions  was  also  investigated.  The  most  illustrative 
of  these  graphs  are  shown  in  Figs.  10-19.  The  flow  chart  for  olotting 
these  graphs  is  shown  in  Fig.  9. 

The  post-training  oerformance  of  the  two  groups  was  analyzed  to 
determine  if  the  fast-time  solutions  would  be  required  to  maintain 
an  acceptable  steady-state  performance  after  the  training  period  was 
comnleted. 
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VI.  RESULTS  OF  DATA  ANALYSIS 

The  two  groups  of  graphs  were  comoared  to  each  other  to  see  how 
the  ODerators  performed.  The  graphs  within  the  groims  were  also  com- 
pared to  see  what  correlations,  if  any,  could  be  derived  from, the 
various  initial  conditions. 

The  curves  plotted  without  regard  to  initial  conditions  were  ex- 
amined first.  The  mean  values  for  the  operators  who  did  not  have  the 
use  of  the  predicted  solutions  were  very  erratic.  They  had  essentially 
learned  the  system  by  run  number  85,  but  even  then  five  of  the  last  15 
runs  had  mean  values  of  greater  than  50(MEAN(K)).  The  largest  mean 
value  was  165  on  run  nine.  The  standard  deviation  varied  even  more 
than  the  mean.  During  the  first  50  runs,  only  24  runs  had  a  standard 
deviation  less  than  50.  During  the  second  50  runs,  there  were  38  runs 
with  a  standard  deviation  less  than  50.  There  were  19  runs  in  which 
the  maximum  error  was  egual  to  or  greater  than  1000.  Seven  of  these 
were  during  the  second  50  runs. 

The  human  operators  who  had  use  of  the  fast-time  solutions  did 
considerably  better  than  their  counterparts.  These  ODerators  had  a 
training  period  of  60  runs.  However,  if  it  had  not  been  for  only  four 
runs  in  which  the  mean  value  was  greater  than  50,  the  training  would 
have  been  accomplished  by  run  30.  The  highest  mean  value  was  145  on 
run  two.  The  standard  deviations  were  not  spasmotic  and  consistently 
improved  throughout  the  100  runs.  There  were  only  three  maximum  errors 
greater  than  60  after  run  number  60.  These  plots  are  illustrated  in 
Figs.  10-19. 
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When  only  the  velocity  challenqe  was  considered,  the  training 
periods  for  the  fast-time  solutions  were  the  same  no  matter  what  the 
initial  velocity  was.  The  mean  value  and  the  standard  deviation  were 
approximately  50  percent  greater  when  there  was  an  initial  velocity 
challenqe.  There  was  an  increase  in  the  training  time  when  no  predicted 
solutions  were  displayed  however.  The  period  was  run  80  with  a  velocity 
challenge  and  60  without  the  challenge. 

Table  1  shows  the  approximate  training  times  for  the  various  com- 
binations of  position  and  velocity  challenges.  In  everv  single  case 
the  training  time  for  the  predicted  solutions  was  less  than  or  egual 
to  the  training  time  of  the  real-time-only  solutions.  The  biqgest 
improvement  was  when  the  initial  position  challenge  was  in  sectors  four 
and  five.  The  improvement  was  a  great  as  70  runs.  There  was  also  a 
very  large  difference  in  the  standard  deviation  in  these  two  sectors, 
the  fast-time  solutions  being  consistently  small  and  the  real-time- 
only  solutions  being  large  and  erratic.  The  plots  of  runs  starting 
from  sector  five  are  shown  in  Figs.  14-19  to  more  clearly  illustrate 
the  difference.  In  general  all  the  fast-time  solution  standard 
deviations  were  considerably  less  than  those  for  the  real-time-only 
solutions,  by  an  approximate  factor  of  three. 

The  steady-state  value  of  the  post-traininq  performance  was  less 
by  a  factor  of  two  for  the  fast-time  operators.  The  performance  for 
these  operators  was  also  very  steady  with  very  little  deviation  in 
the  mean  value  of  the  error.  This  was  not  true  for  the  real-time- 
only  operators. 
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VII.  CONCLUSIONS 

The  human  operators  who  had  the  use  of  Dredictive  solutions  were 
able  to  learn  the  control  system  sooner  than  operators  who  did  not 
have  the  aid  of  such  solutions.  The  decreased  traininq  time  was  even 
more  pronounced  when  the  initial  challenge  was  greater. 

This  leads  to  the  conclusion  that  the  training  of  human  operators 
can  be  greatly  enhanced  with  predictive  solutions.  The  decrease  in 
improvement  of  the  training  period  when  the  initial  challenge  was  a 
lesser  degree  shows  that  for  the  predictive  solutions  to  be  worthwhile, 
the  system  must  indeed  be  comolex  and  slow  in  responding  to  the  input. 

The  improvement  in  the  steady-state  performance  when  the  predictive 
solutions  were  provided,  indicates  that  the  use  of  predictive  solutions 
in  production  runs  might  be  necessary  in  some  cases.  When  consistency 
and  precise  accuracy  are  needed,  the  predictive  solutions  would  nrove 
to  be  of  value  even  after  training  is  completed.  However,  if  exoense 
and  practicality  are  considered  to  be  more  important  than  precise 
performance,  the  fast-time  solutions  would  not  be  needed.  These  con- 
clusions depend  on  the  system  under  consideration  and  no  exact  rule  for 
the  use  of  predictive  solutions  in  productive  operation  can  be  drawn 
without  further  investigation  in  this  area. 
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APPENDIX  A 

1.   DISPLAY  CIRCUITRY 

The  oscilloscope  used  for  disDlay  to  the  human  operator  had  only 
sinqle  X  and  Y  inputs.  In  order  to  display  two  separate  pieces  of 
information  simultaneously,  it  was  necessary  to  use  four  diqital-to- 
analoq  switches,  summing  junctions  of  two  amplifiers,  and  two  hiqh-speed 
delay  flops.  Two  more  hiqh-speed  delay  flops  were  input  to  a  nand 
qate  and  its  output  went  to  the  sweep-blank  input  of  the  oscilloscope. 
This  eliminated  the  presence  of  a  continuous  trace  line  between  the 
two  solution  spots  on  the  scooe  face.  The  circuitry  is  shown  in 
Figs.  2  and  5. 
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